Skip to content

daemon: allow configuring initial interests#732

Closed
m0ar wants to merge 1 commit into
ceramicnetwork:mainfrom
m0ar:m0ar/configure-startup-interests
Closed

daemon: allow configuring initial interests#732
m0ar wants to merge 1 commit into
ceramicnetwork:mainfrom
m0ar:m0ar/configure-startup-interests

Conversation

@m0ar

@m0ar m0ar commented Aug 14, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

This PR implements a new CLI flag --initial-interests (and envvar CERAMIC_ONE_INITIAL_INTERESTS) to automatically register node interests when starting the daemon, eliminating the need for manual API calls after startup.

Changes

  • New CLI argument --initial-interests, accepting comma-separated stream/model IDs
  • New startup module one/src/startup.rs to orchestrate daemon configuration tasks tangential to the startup process
  • Integration:
    • Uses the existing InterestService
    • Uses the same pattern as the /ceramic/interests API handler
    • Processes interests early in daemon startup, after services init but before P2P setup

Safety & backward compatibility

  • Optional flag with empty default
  • Stream ID format and multibase decoding validation
  • Descriptive errors for invalid inputs + handling of failures
  • Logging for registration status and debugging
  • Idempotent: Handles already-registered interests without errors

Usage

Flag

ceramic-one daemon --initial-interests model1,model2

Envvar

export CERAMIC_ONE_INITIAL_INTERESTS="model1,model2"
ceramic-one daemon

Tests

Test suite added to one/src/startup.rs, can be run with the C1 crate:

> cargo test --locked --release -p ceramic-one
[ build logs ]

running 8 tests
test http::test::expected_prom_cardinality_replacement ... ok
test http::test::should_not_replace_extra_parts ... ok
test http::test::should_not_replace_wrong_method ... ok
test startup::tests::test_process_initial_interests_invalid_stream_id ... ok
test startup::tests::test_process_initial_interests_empty ... ok
test startup::tests::test_process_initial_interests_multibase_decoding ... ok
test startup::tests::test_process_initial_interests_mixed_valid_invalid ... ok
test startup::tests::test_process_initial_interests_whitespace_handling ... ok

test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

Other notes

It seems to work, but I can't say I fully understand what's going on with the interest builder EventID fencepost ranges. Probably a good spot to focus a review 👀

Closes #707

@m0ar m0ar requested a review from a team as a code owner August 14, 2025 11:23
@m0ar m0ar requested review from smrz2001 and removed request for a team August 14, 2025 11:23
@m0ar m0ar force-pushed the m0ar/configure-startup-interests branch 2 times, most recently from 556f8cd to 1f6ea28 Compare August 14, 2025 11:26
@stbrody stbrody requested a review from dav1do August 19, 2025 21:21
dav1do
dav1do reviewed Aug 22, 2025
View reviewed changes

@dav1do dav1do left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I have a few small things/questions but nothing blocking.

Comment thread one/src/daemon.rs Outdated
value_delimiter = ',',
env = "CERAMIC_ONE_INITIAL_INTERESTS"
)]
initial_interests: Vec<String>,

@dav1do dav1do Aug 22, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: trying to decide what this name should be.. it's not exactly "initial" because they don't ever go away, but NEW or MODEL steams a bit silly

@dav1do dav1do Aug 22, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, I wonder if we want to subscribe to the METAMODEL_STREAM_ID automatically/easily? unless we do and I missed it, but that's the "I'd like to see all models on the network".

@m0ar m0ar Aug 26, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EXTRA_INTERESTS kinda matches how the daemon treats "appending" peers, so I'll go with that 👌

@m0ar m0ar Aug 26, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, I wonder if we want to subscribe to the METAMODEL_STREAM_ID automatically/easily? unless we do and I missed it, but that's the "I'd like to see all models on the network".

Myeah, but for testing scenarios it's also pretty useful to have an "idle" node by default. I'll leave automatic MM interest registration out for now, but add the MM id to the config docs on this flag.

Comment thread one/src/startup.rs
let result =
process_initial_interests(&stream_ids, &interest_svc, &network, &node_id).await;
// Should succeed with proper multibase decoding
assert!(result.is_ok());

@dav1do dav1do Aug 22, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe add an assertion this was written to the db?

let vals = interest_svc.full_range().await.unwrap();
assert!(vals.count(), 1);

Comment thread one/src/startup.rs
let stream_id_bytes = multibase::decode(stream_id_str)
.map_err(|e| anyhow!("Failed to decode stream ID '{}': {}", stream_id_str, e))?
.1;
let start = EventId::builder()

@dav1do dav1do Aug 22, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was considering if it'd be worth putting this in interest service or core so we could share most of this with the http server. But I think this API is fine to be exposed, I just couldn't remember how to build interest ranges and wanted a higher level API that's like subscribe_to_model 😅

@m0ar m0ar Aug 26, 2025

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I agree, but it's not entirely clear how to do that so I'll leave it for now

@m0ar m0ar force-pushed the m0ar/configure-startup-interests branch from 1f6ea28 to ba5e12f Compare August 26, 2025 13:16
@m0ar m0ar force-pushed the m0ar/configure-startup-interests branch from ba5e12f to 8c19950 Compare August 26, 2025 13:57
@m0ar

m0ar commented Aug 26, 2025

Copy link
Copy Markdown
Collaborator Author

Closed in favour of #734

@m0ar m0ar closed this Aug 26, 2025
@m0ar m0ar temporarily deployed to tnet-prod-2024 August 26, 2025 14:27 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dav1do dav1do dav1do left review comments

@smrz2001 smrz2001 Awaiting requested review from smrz2001 smrz2001 is a code owner automatically assigned from ceramicnetwork/eng-reviewers

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add --initial-interests configuration variable

2 participants

@m0ar @dav1do